fix: preserve non-ASCII characters in JSON serialization #1512
+2
−0
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add ensure_ascii=False to json.dumps calls to prevent Chinese, Japanese,
and other non-ASCII characters from being escaped as \uXXXX sequences.
This fixes the display of non-ASCII text in trace list views, where
characters were showing as Unicode escape sequences instead of
readable text.
Important
Add
ensure_ascii=Falseto JSON serialization inutils.pyandserializer.pyto preserve non-ASCII characters.ensure_ascii=Falsetojson.dumpsinspan_formatter()inutils.pyto prevent non-ASCII characters from being escaped.ensure_ascii=Falsetosuper().encode()inencode()inserializer.pyto handle non-ASCII characters correctly.This description was created by
for 0a38cb5. You can customize this summary. It will automatically update as commits are pushed.
Disclaimer: Experimental PR review
Greptile Overview
Greptile Summary
This PR aims to preserve non-ASCII characters (Chinese, Japanese, etc.) in JSON serialization by adding
ensure_ascii=Falseto prevent Unicode escape sequences.Critical Issue Found:
langfuse/_utils/serializer.py:176contains a syntax error that will causeTypeErrorat runtimeJSONEncoder.encode()does not acceptensure_asciias a parameter - it must be set in the constructor insteadlangfuse/_client/utils.py:62is correctImpact:
EventSerializer().encode()will fail withTypeErrorlangfuse/_utils/request.py:63), score ingestion (langfuse/_task_manager/score_ingestion_consumer.py:88, 120), and attribute conversion (langfuse/_client/attributes.py:177)Required Fix:
Set
ensure_ascii=FalseinEventSerializer.__init__()instead of passing it tosuper().encode()Confidence Score: 0/5
langfuse/_utils/serializer.py:176passesensure_ascii=Falsetosuper().encode(), but Python'sJSONEncoder.encode()method only accepts one parameter (the object to encode). This will cause aTypeError: encode() takes 2 positional arguments but 3 were givenwhen the code runslangfuse/_utils/serializer.pyrequires immediate correction to prevent runtime errorsImportant Files Changed
ensure_ascii=Falsetojson.dumps()call inspan_formatter()to preserve non-ASCII charactersensure_ascii=Falsetosuper().encode()but this parameter is not accepted by the method - will cause TypeErrorSequence Diagram
sequenceDiagram participant Client participant json.dumps() participant EventSerializer participant JSONEncoder Note over Client,JSONEncoder: Two usage patterns rect rgb(200, 220, 240) Note over Client,EventSerializer: Pattern 1: Using json.dumps with cls parameter Client->>json.dumps(): json.dumps(data, cls=EventSerializer) json.dumps()->>EventSerializer: __init__() with ensure_ascii EventSerializer->>JSONEncoder: super().__init__(ensure_ascii=False) json.dumps()->>EventSerializer: default(obj) EventSerializer-->>json.dumps(): processed obj json.dumps()-->>Client: JSON string with non-ASCII chars end rect rgb(240, 220, 200) Note over Client,EventSerializer: Pattern 2: Direct instantiation Client->>EventSerializer: serializer = EventSerializer() EventSerializer->>JSONEncoder: super().__init__(ensure_ascii=False) Client->>EventSerializer: serializer.encode(obj) EventSerializer->>EventSerializer: self.default(obj) EventSerializer->>JSONEncoder: super().encode(processed_obj) Note over EventSerializer,JSONEncoder: BUG: ensure_ascii passed here (line 176)<br/>but encode() doesn't accept parameters JSONEncoder-->>EventSerializer: JSON string EventSerializer-->>Client: JSON string end(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!